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I.  Summary 

The  overall  objectives  of  this  research  were  to  analyze  the  properties  of  identified 
neurons  and  neural  circuits  in  Aplysia  that  exhibit  nonassociative  and  associative  plasticity 
and  to  examine  the  role  of  neuronal  plasticity  in  learning.  Two  interrelated  approaches 
were  used,  one  empirical  and  the  other  modeling.  Between  August  1, 1987  and 
September  30, 1990,  progress  was  made  in  six  areas.  First,  experiments  examined  the 
modulation  of  membrane  currents  and  critical  second  messengers  that  contribute  to 
neuronal  plasticity  in  the  sensory  neurons  that  mediate  nonassociative  learning  and  classical 
conditioning  of  the  tail  withdrawal  reflex.  Second,  mathematical  formalisms  of  the  cellular 
processes  that  underlie  plasticity  in  sensory  neurons  were  developed  and  incorporated  into  a 
real-time  model  of  a  neuron-like  adaptive  element.  This  single  cell  model  demonstrated  the 
ability  to  simulate  features  of  nonassociative  learning  and  classical  conditioning.  Third,  this 
single  cell  model  was  incorporated  into  small  neural  networks  that  simulated  some  higher- 
order  features  of  classical  conditioning.  Fourth,  a  form  of  the  single  cell  model  was 
incorporated  into  a  small  pattern  generating  neural  network  that  simulated  features  of 
operant  conditioning.  Fifth,  a  Hodgkin-Huxley  type  of  model  of  biophysical  and  cellular 
processes  underlying  rhythmic  bursting  patterns  of  activity  in  the  neuron  R15  was 
developed.  Sixth,  experiments  have  identified  neurons  that  are  elements  of  the  central 
pattern  generator  that  controls  feeding  behavior  and  that  may  mediate  operant  conditioning 
of  feeding  behavior. 


II.  Research  Objectives 

The  research  that  was  proposed  in  AFOSR  Grant  87-0274  had  four  specific  aims:  1)  to 
analyze  the  membrane  properties  of  sensory  neurons  and  to  incorporate  this  information 
into  our  neuron-like  model  of  learning,  2)  to  examine  the  ability  of  our  neuron-like  model 
of  learning  to  simulate  higher-order  features  of  classical  conditioning,.  3)  to  analyze  the 
properties  of  facilitatory  and  inhibitory  interneurons  that  might  contribute  to  associative 
learning,  and  4)  to  develop  a  cellular  preparation  that  is  amenable  to  the  cellular  analysis  of 
operant  conditioning,  and  to  simulate  features  of  operant  conditioning  in  small  neural 
networks.  Two  interrelated  approaches  were  used,  one  empirical  and  the  other  modeling. 
The  empirical  approach  utilized  cellular  neurophysiological  techniques  in  order  to  analyze 
the  particular  ionic  conductances  and  cellular  processes  that  underlie  forms  of  neuronal 
plasticity.  The  modeling  approach  involved  formulating  mathematical  descriptions  of  the 
neuronal  mechanisms  and  neural  circuits,  and  examining  whether  simulation  of  the  resultant 
model  simulated  the  empirical  data  and  features  of  learning.  The  two  approaches  were 
interrelated  in  that  empirical  data  were  used  to  constantly  test  and  improve  the  quality  of 
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the  models,  and  predictions  of  the  models  were  used  to  guide  the  design  of  new 
experiments. 


III.  Status  of  Research  (Progress  Report) 

Progress  during  the  past  38  months  has  been  in  six  areas.  First,  the  experimental 
analysis  of  the  cellular  mechanisms  contributing  to  neuronal  plasticity.  Second,  the 
development  of  quantitative,  real-time  models  of  neuron-like  adaptive  elements  tha* 
simulate  simple  features  of  nonassociative  and  associative  learning.  Third,  the 
implementation  of  our  neuron-like  model  for  learning  into  small  networks  and  simulation  of 
higher-order  features  of  classical  conditioning.  Fourth,  the  simulation  of  features  of  operant 
conditioning  with  a  neuron-like  model  of  learning  that  was  originally  derived  in  order  to 
simulate  classical  conditioning.  Fifth,  the  development  of  a  quantitative,  real-time  model  of 
the  bursting  neuron  R15.  Sixth,  identification  of  neuronal  elements  of  the  central  pattern 
generator  that  controls  feeding  behavior  and  that  may  contribute  to  operant  conditioning  of 
feeding  behavior. 


A.  Experimental  Analysis  of  the  Modulation  of  Membrane  Currents  and  Critical  Second 
Messengers  that  Contribute  to  Neuronal  Plasticity  in  Tail  Sensory  Neurons 


1.  Serotonergic  Modulation  of  Two  Potassium  Currents  in  Pleural  Sensory  Neurons  of 
Aplysia.  The  serotonergic  modulation  of  membrane  currents  is  an  important 
mechanism  contributing  to  presynaptic  facilitation  of  transmitter  release  from  sensory 
neurons,  which  in  turn,  is  thought  to  be  a  cellular  mechanism  contributing  to  several 
simple  forms  of  learning.  Previous  analyses  of  the  biophysical  mechanisms  of  plasticity 
in  the  sensory  neurons  of  Aplysia  have  focused  on  a  serotonin  (5-HT)-sensitive  K+ 
current,  which  is  termed  the  S  current  (/K  s).  Application  of  5-HT  reduces  the 
magnitude  of  /Ks  via  cAMP-dependent  protein  phosphorylation  that  closes  the 
channel. 


1  cwpt 

\f*»si»€crto  f 


We  demonstrated  a  novel  action  of  5-HT  on  membrane  currents  in  somata  of 
sensory  neurons.  The  modulatory  effects  of  5-HT  on  the  membrane  currents  were 
revealed  by  computer  subtraction  of  current  responses  elicited  in  the  presence  of 
5-HT  from  current  responses  elicited  prior  to  the  application  of  5-HT.  The 
complexities  of  the  resulting  5-HT  difference  currents  (I^j)  suggested  that  5-HT 
modulated  at  least  two  components  of  membrane  current.  At  membrane  potentials 
more  negative  than  0  mV,  application  of  5-HT  suppressed  a  component  of  membrane 
current  that  had  properties  identical  to  those  previously  found  for  7j^s  .  This  relatively 
voltage-independent  component  of  /5.Hj  was  active  at  or  near  the  resting  potential, 
did  not  inactivate,  increased  in  a  time-dependent  manner  during  depolarization,  was 
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relatively  insensitive  to  the  K+  channel  blockers  4-aminopyridine  (4-AP)  and 
tetraethylammonium  (TEA),  and  was  suppressed  by  bath  application  of  cAMP.  At 
membrane  potentials  more  positive  than  OmV,  5-HT  modulated  an  additional 
component  of  membrane  current  that  had  properties  characteristic  of  the  delayed  or 
voltage-dependent  K+  current  (/^y)-  This  component  had  complex  kinetics,  was 
highly  voltage-dependent,  and  was  blocked  by  moderate  concentrations  of  either  4-AP 
or  TEA.  We  concluded  that  5-HT  modulated  y ,  possibly  by  slowing  its  kinetics  of 
activation  and  inactivation.  These  results  indicate  that  the  modulation  of  membrane 
currents  by  5-HT  is  more  complex  than  was  originally  thought.  Given  that  more  than 
one  membrane  current  is  modulated,  a  key  issue  becomes  the  quantitative  extent  to 
which  modulation  of  specific  currents  contributes  to  modulation  of  spike  parameters 
and  plasticity  of  the  sensory  neurons  (see  Sections  III.A.2.  and  III.B.3). 

2.  Differential  Effects  of  cAMP  and  Serotonin  on  Membrane  Currents,  Action  Potential 
Duration,  and  Excitability  in  Pleural  Sensory  Neurons  of  Aplysia.  Since  5-HT 
modulates  more  than  one  component  of  membrane  current,  an  additional  question  is 
whether  modulation  of  the  second  component  (/j^v)  is  mediated  via  cAMP  as  is  the 
first  (/K  s).  In  order  to  determine  whether  the  modulation  of  /j^v  by  5-HT  is  also 
mediated  via  cAMP,  we  compared  the  effects  of  a  membrane-permeable  and 
phosphodiesterase-resistant  analogue  of  cAMP  on  membrane  currents  with  those  of 
5-HT.  At  all  voltage-clamp  potentials  examined  (-40  to  -1-30  mV),  the  addition  of 
cAMP  to  the  bath  reduced  the  amplitude  of  the  membrane  current  at  the  end  of  the 
voltage-clamp  pulse.  The  cAMP-sensitive  component  of  membrane  current  was 
relatively  voltage-independent,  increased  slowly  during  the  voltage-clamp  pulse,  did 
not  inactivate,  and  was  active  at  or  near  the  resting  potential.  These  properties  are 
consistent  with  those  of  /Ks  ,  but  are  not  indicative  of  a  modulation  of  /K  y  by  cAMP. 
Indeed,  the  presence  of  cAMP  did  not  occlude  modulation  by  5-HT  of  /KV  .  These 
results  indicated  that  /K  s  is  modulated  by  cAMP,  whereas  /K  v  is  not. 

In  order  to  gain  insight  into  the  relative  contributions  of  cAMP-dependent  and 
cAMP-independent  components  to  the  serotonergic  modulation  of  the 
electrophysiological  properties  of  sensory  neurons,  we  first  examined  the  effects  of 
cAMP  on  the  duration  of  action  potentials  and  neuronal  excitability  and  then 
examined  the  effects  of  the  subsequent  addition  of  5-HT  to  the  bath,  which  still 
contained  cAMP.  The  application  of  cAMP  increased  the  duration  of  the  action 
potential  by  an  average  of  24%.  The  subsequent  addition  of  5-HT  to  the  bath,  which 
still  contained  cAMP,  dramatically  increased  the  duration  of  the  action  potential  to  an 
average  of  234%  of  the  control  duration.  Thus,  a  cAMP-independent  modulation  of 
membrane  current  by  5-HT  appeared  to  be  critical  for  regulating  the  duration  of  the 
action  potential.  In  contrast  to  a  modest  enhancement  of  the  duration  of  the  action 
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potential  produced  by  cAMP,  application  of  cAMP  doubled  the  number  action 
potentials  elicited  during  a  1  s,  2  nA  current  pulse.  The  subsequent  addition  of  5-HT 
to  the  bath,  which  still  contained  cAMP,  failed  to  produce  any  further  increase  in 
excitability.  Thus,  the  presence  of  cAMP  appeared  to  occlude  any  further  modulation 
of  neuronal  excitability  by  the  subsequent  addition  of  5-HT. 

B.  Mathematical  Models  and  Simulations  of  Cellular  Processes  Underlying  Neuronal 

Plasticity  in  Sensory  Neurons 

1.  Single  Cell  Model  of  Associative  Learning.  One  of  the  useful  aspects  of  the  Aplysia 
system  has  been  the  ability  to  combine  modeling  approaches  with  empirical  ones. 
Indeed,  predictions  from  the  models  have  led  to  the  discovery  of  additional  processes 
contributing  to  presynaptic  facilitation.  Our  approach  has  been  to  transform  the 
various  biochemical  and  biophysical  mechanisms  that  contribute  to  nonassociative  and 
associative  plasticity  within  the  sensory  neurons  into  mathematical  formalisms,  assign 
values  to  the  parameters  that  agree  with  published  data,  and  fit  the  components 
together  to  create  a  model  of  transmitter  release  at  the  sensory  to  motor  synapse. 

Briefly,  the  model  includes  differential  equations  describing  two  pools  of 
transmitter,  a  releasable  pool  (PR)  and  a  storage  pool  (Ps).  Vesicles  of  transmitter 
are  mobilized  from  the  storage  pool  to  the  releasable  pool  via  three  fluxes,  one  driven 
by  diffusion  (FD),  one  driven  by  Ca2+  (Fc),  and  one  driven  by  cAMP  (FcAMP).  The 
storage  pool  is  replenished  by  the  synthesis  of  new  vesicles  (FN).  Differential 
equations  also  describe  the  regulation  of  intracellular  levels  of  both  Ca2+  and  cAMP. 
Simulated  action  potentials  (all-or-nothing  3  ms  depolarizing  pulses)  lead  to  the  influx 
of  Ca2+  ( Iq a)  into  an  intracellular  pool  (Ca2+  Pool),  the  Ca2+-mediated  release  of 
transmitter  (TR),  and  Ca2+-mediated  mobilization  of  transmitter.  Sensitizing  or 
reinforcing  stimuli  (US)  lead  to  the  release  of  facilitatory  transmitters,  which  in  turn 
lead  to  the  synthesis  of  cAMP,  the  mobilization  of  transmitter,  and  to  an  increase  in 
the  duration  of  the  action  potential;  thus  allowing  greater  Ca2+  influx  and  enhanced 
release  of  transmitter.  The  effect  of  the  conditioned  stimulus  (CS)  (Ca2+  entry  during 
spikes  in  the  sensory  neuron)  on  levels  of  cAMP  is  simulated  by  making  the 
concentration  of  cAMP  dependent  on  the  intracellular  concentration  of  Ca2+.  This 
dual  regulation  of  the  cyclase  by  the  modulatory  transmitter  and  the  CS  (Ca2+  entry) 
permits  the  formation  of  an  association  between  the  ‘neuronal  trace’  of  the  CS  and  the 
unconditioned  stimulus  (US). 

Our  simulations  have  illustrated  that  this  neuron-like  element  accurately  simulates 
many  aspects  of  the  nonassociative  plastic  properties  of  the  sensory  neurons.  For 
example,  repetitive  action  potentials  in  the  model  result  in  the  cumulative  inactivation 


Page  5 


AFOSR  Grant  87-0274 


John  H.  Byrne 


of  Iq^  ,  depletion  of  transmitter  in  PR  ,  and  thus,  depression  of  synaptic  transmission. 
This  synaptic  depression  is  the  neuronal  analogue  of  habituation.  In  addition,  the 
single-cell  model  accurately  simulates  presynaptic  facilitation,  which  is  the  neuronal 
analogue  of  dishabituation  and  sensitization. 

This  neuron-like  element  also  simulates  many  aspects  of  associative  plastic 
properties  of  sensory  neurons,  which  are  believed  to  be  cellular  mechanisms 
contributing  to  classical  conditioning.  Moreover,  it  predicted  an  interstimulus  interval 
(ISI)  function  very  similar  to  that  observed  behaviorally  in  Aplysia  and  other  animals. 
The  simulated  ISI  function  in  the  neuron-like  element  is  a  direct  consequence  of  the 
kinetics  of  the  buffering  of  intracellular  Ca2+.  When  the  ISI  is  short,  high  levels  of 
Ca2+  are  present  at  the  time  of  the  US  and  therefore  the  CS-mediated  amplification 
of  the  effects  of  the  US  are  greatest.  As  the  interval  between  the  CS  and  US 
increases,  Ca2+  levels  are  buffered;  consequently  with  longer  ISIs,  there  is  less 
amplification  of  the  effects  of  the  US.  Thus,  the  elevation  of  intracellular  Ca2+  levels 
produced  by  the  CS  serves  as  a  "trace"  that  becomes  associated  with  a  closely  paired 
US.  The  higher  levels  of  cAMP  result  in  greater  presynaptic  facilitation. 

2.  Mathematical  Model  of  Cellular  Mechanisms  Contributing  to  Presynaptic  Facilitation. 
Presynaptic  facilitation  of  transmitter  release  from  sensory  neurons  is  an  important 
mechanism  contributing  to  nonassociative  and  associative  learning  in  Aplysia.  As 
described  above  (Section  III.B.l.)  we  concluded  that  enhancement  of  the  postsynaptic 
potential  (PSP)  during  presynaptic  facilitation  is  mediated  by  at  least  two  processes; 
spike  broadening,  which  has  been  observed  experimentally,  and  a  process  that  we 
modeled  as  mobilization  of  transmitter.  In  an  effort  to  gain  insight  into  the  relative 
contribution  of  these  two  mechanisms  of  presynaptic  facilitation,  we  have  extended 
our  earlier  model  to  include  more  detailed  descriptions  of:  a)  the  kinetics  of  the  Ca2+ 
channel,  b)  the  diffusion  of  Ca2+  through  the  cytoplasm,  c)  the  process  of  transmitter 
release,  and  d)  the  PSP.  This  extended  model  provides  an  accurate  description  of  the 
input-output  relationship  for  synapses  of  sensory  neurons,  and  predicts  changes  in  the 
shape  of  postsynaptic  potentials  as  a  function  of  mobilization  and  spike  broadening. 
The  results  confirm  and  extend  previous  experimental  studies  and  indicated  that 
cellular  analogs  of  sensitization  (facilitation  of  nondecremented  responses)  is 
mediated  primarily  by  spike  broadening;  whereas,  analogs  of  dishabituation 
(facilitation  of  depressed  of  responses)  require  mobilization. 

3.  Mathematical  Model  of  Serotonergic  Modulation  of  Electrophysiological  Properties 
of  Sensory  Neurons.  The  voltage-  and  current-clamp  experiments  described  above, 
Section  III.  A.,  indicate  that  the  effects  of  5-HT  on  the  duration  of  the  action  potential 
and  on  the  excitability  in  sensory  neurons  is  due  to  the  modulation  of  at  least  two 
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membrane  currents,  /j^s  and  /j^v.  In  order  to  assess  the  quantitative  contribution 
that  each  current  makes  to  the  overall  modulatory  effects  of  5-HT,  we  constructed  a 
Hodgkin-Huxley  type  membrane  model  of  the  somata  of  the  sensory  neuron.  The 
model  consists  of  a  membrane  capacitance,  a  leakage  conductance  with  its  associated 
equilibrium  potential,  and  differential  equations  that  describe  eight  ionic  currents  with 
their  associated  conductances  and  equilibrium  potentials:  the  Na+  current  (/Na ),  a 
Ca2+  current  with  a  high  threshold  component  (/^  N  )  and  low  threshold  component 
(^Ca  L )»  ^k,S  ’  Ac,V » *be  fast  K+  current  (/A ),  and  a  Ca2+-activated  K+  current  with  a 
slow  component  )  and  a  fast  component  (/7K  ^ ).  In  addition,  the  model 

includes  a  description  of  intracellular  Ca2+  concentrations.  Parameter  values  for 
many  of  the  current  were  estimated  from  voltage-clamp  data  from  sensory  neurons, 
while  values  for  the  remaining  parameter  were  adjusted  to  qualitatively  simulate 
features  of  excitability  and  spikes  in  sensory  neurons. 

One  action  of  cAMP  is  to  initiate  the  closure  of  S-channels,  which  results  in  a 
reduction  in  the  amplitude  /j^s  .  Thus  we  simulated  the  cAMP-dependent  actions  of 
5-HT  on  membrane  current  by  reducing  the  maximum  conductance  of  /KS  in  the 
model.  Although  the  mechanism  underlying  the  cAMP-independent  modulation  of 
/K  v  by  5-HT  is  not  yet  known,  we  have  speculated  that  5-HT  may  lead  to  a  slowing  in 
the  rates  of  activation  and  inactivation  of  /^y.  Thus  we  simulate  the  cAMP- 
independent  actions  of  5-HT  on  membrane  current  by  altering  the  voltage-dependent 
time  constants  for  activation  and  inactivation  of  I^r  v  in  the  model.  The  simulated 
membrane  currents  and  their  modulation  by  cAMP  and  5-HT  were  similar  to  those 
observed  experimentally. 

Given  a  reasonable  fit  of  the  normal  electrophysiological  properties  of  sensory 
neurons,  we  then  adjusted  the  parameters  of  those  membrane  currents  that  are 
modulated  by  5-HT  separately  to  try  to  gain  insights  into  the  specific  role  they  play  in 
regulating  spike  broadening  and  excitability.  Reducing  only  /KS  in  the  model 
produced  a  modest  (10%)  broadening  of  the  simulated  spike.  In  contrast,  modifying 
IK  v  produced  a  dramatic  increase  in  spike  duration  (200%).  While  reducing  /K  s  did 
not  significantly  increase  spike  duration,  it  did  produce  a  significant  increase  in 
excitability.  In  contrast,  simulations  in  which  /K  v  was  also  altered  did  not  show  any 
further  increase  in  excitability. 

These  initial  simulations  must  be  repeated  after  more  detailed  information  on  the 
properties  of  the  sensory  neuron  membrane  currents  is  available.  Nevertheless,  the 
results  of  these  simulation  are  remarkably  similar  to  the  physiological  actions  of  cAMP 
and  5-HT.  These  simulations  support  the  hypothesis  that  the  cAMP-dependent 
modulation  /K  s  by  5-HT  plays  a  key  role  in  regulating  excitability  of  sensory  neurons, 
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while  the  cAMP-independent  modulation  of  /j^v  by  5-HT  appears  to  be  critical  for 
regulating  the  duration  of  the  action  potential. 

C.  Empirically  Derived  Neuron-Like  Elements  and  Neural  Networks  Simulate  Higher- 
Order  Features  of  Classical  Conditioning 

As  described  above  (Section  III.B.l.)  we  developed  a  model  of  a  neuron-like  element 
that  incorporates  descriptions  of  cellular  processes  that  contribute  to  plasticity  of  the 
sensory  neurons.  Our  simulations  have  illustrated  that  this  neuron-like  element  is 
capable  of  predicting  features  of  nonassociative  learning  and  simple  features  of  classical 
conditioning.  We  extended  these  observations  by  examining  the  ability  of  small  neural 
networks  that  include  this  neuron-like  element  to  predict  more  complex  features  of 
associative  learning,  such  as  higher-order  features  of  classical  conditioning. 

We  implemented  two  small  neural  networks  that  are  derived  from  the  characteristics 
of  the  neural  circuit  mediating  withdrawal  reflexes  of  Aplysia.  In  each  network,  two 
sensory  neurons  (SN1  and  SN2)  with  identical  properties  make  synaptic  contact  with  a 
motor  neuron  (MN).  Activity  in  the  individual  sensory  neurons  represents  separate 
pathways  for  conditioned  stimuli  (CS1  and  CS2).  The  amplitude  of  the  EPSPs  at  the 
sensory-to-motor  synapses  represent  the  ability  of  each  CS  to  activate  the  response 
system  and  produce  a  conditioned  response  (CR).  A  key  aspect  of  the  architecture  of 
this  network  is  that  the  sensory  neurons  also  make  excitatory  synaptic  connections  with  a 
facilitatory  interneuron  (FN).  This  connection  has  two  consequences,  one  practical  and 
the  second  theoretical.  First,  from  a  practical  point  of  view,  the  circuit  can  be  simplified 
by  removing  the  motor  neuron  and  using  the  EPSP  at  the  sensory-to-facilitatory  synapse 
as  measure  of  changes  in  the  synaptic  strength  of  the  CS.  Second,  from  a  theoretical 
point  of  view,  a  sensory  neuron  can  ‘take  control’  of  the  facilitatory  interneuron  as  the 
strength  of  its  sensory-to-facilitatory  synapse  increases.  This  possibility  has  fundamental 
implications  with  respect  to  neural  models  of  higher-order  features  of  classical 
conditioning.  Because  there  is  little  experimental  data  on  the  properties  of  the 
facilitatory  neuron,  it  is  modeled  as  an  element  that  simply  sums  its  synaptic  inputs,  and  if 
the  inputs  are  equal  to  or  exceed  a  threshold,  a  simulated  action  potential  is  initiated.  An 
important  assumed  property  of  the  facilitatory  neuron  is  that  its  output  diminishes  or 
accommodates  rapidly,  and  that  its  recovery  from  accommodation  is  relatively  slow. 
Normally,  the  presentation  of  the  unconditioned  stimulus  (US)  always  activates  the 
facilitatory  interneuron  and  stimulates  the  release  of  the  facilitatory  transmitter,  which  in 
turn  activates  an  adenylate  cyclase  in  the  sensory  neuron. 

In  a  preliminary  attempt  to  develop  a  network  that  incorporates  recently  described 
features  of  inhibition,  we  implemented  a  five-cell  network  that  incorporates  inhibitory 
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interneurons.  As  a  first  step,  we  assumed  that  the  inhibitory  interneurons  only  act  on 
cellular  processes  mediating  associative  plasticity  in  the  sensory  neurons.  This  form  of 
inhibition  is  a  hypothetical  mechanism  that  could  occur  at  numerous  loci,  such  as 
blocking  the  influx  of  Ca2+  or  blocking  the  Ca2+  priming  component  of  cAMP  synthesis. 
In  this  simulation,  we  assumed  that  activity  of  the  inhibitory  neuron  transiently  blocks 
further  Ca2+  priming  of  cAMP  synthesis  in  the  sensory  neuron,  but  not  priming  that 
results  from  Ca2+  present  before  the  onset  of  activity  of  the  inhibitory  neuron.  We  also 
assumed  that  the  inhibitory  neuron  has  a  burst-like  property.  The  duration  of  activity  in 
the  inhibitory  neuron  was  a  function  of  the  amount  of  time  its  input  remained  above 
threshold.  The  output  of  the  inhibitory  neuron  was  constant  during  its  activation. 

When  the  single-cell  model  was  incorporated  into  the  three-cell  network,  we  found 
that  the  constraints  imposed  by  the  empirical  data  limited  the  ability  of  the  network  to 
simulate  second-order  conditioning  and  blocking.  On  the  other  hand,  we  found  that  a 
detailed  description  of  subcellular  processes  unmasked  phenomena  that  were  relevant  to 
the  simulation  of  blocking  that  were  not  captured  by  less  detailed  models.  The  tive-cell 
network  successfully  simulated  both  second-order  conditioning  and  blocking  more  readily 
than  the  three-cell  network.  These  simulations  demonstrated  that  neuron-like  elements 
and  network  architectures  that  reflect  the  cellular  processes  contributing  to  activity- 
dependent  neuromodulation  can  simulate  higher-order  features  of  classical  conditioning. 

D.  A  Classical  Conditioning  Learning  Rule  Simulations  Features  of  Operant 
Conditioning  in  a  Pattern  Generating  Neural  Network 

Despite  operational  distinctions,  it  is  not  known  whether  the  cellular  processes 
underlying  classical  conditioning  and  operant  conditioning  are  fundamentally  different  or 
whether  they  share  aspects  of  a  common  underlying  mechanism.  Recent  theoretical 
studies  indicate  that  the  same  neural  networks  and  learning  rules  that  can  simulate 
features  of  classical  conditioning  can  also  simulate  elementary  forms  of  operant 
conditioning.  Therefore,  we  were  interested  in  determining  whether  a  small  network 
containing  elements  with  the  activity-dependent  neuromodulation  learning  rule,  which 
simulates  features  of  classical  conditioning,  could  also  simulate  features  of  operant 
conditioning.  In  order  to  test  this  hypothesis,  we  took  the  mathematical  model  of  the 
sensory  neurons  in  Aplysia  and  incorporated  this  neuron-like  element  into  a  small  neural 
network  and  examined  its  ability  to  simulate  features  of  operant  conditioning. 

Our  simulations  of  operant  conditioning  were  closely  based  on  operant  conditioning 
of  head-waving  behavior  in  Aplysia.  Head-waving  in  Aplysia  is  a  naturally  occurring 
behavior  in  which  the  animals  sweep  their  heads  from  side  to  side.  Aplysia  can  be 
operantly  conditioned  to  wave  their  heads  preferentially  to  one  side  of  their  body. 
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Although  the  neuronal  elements  and  circuit  underlying  this  behavior  are  not  yet  known, 
wl  developed  an  artificial  neural  network  that  produced  a  pattern  similar  to  the  side-to- 
side  movement  in  head-waving.  This  network  contained  three  classes  of  elements: 
pattern  generating  elements  (PGs),  associative  elements  (AEs),  and  motor  neurons 
(MNs).  The  pattern  generating  elements  produced  the  spontaneous  behavior  that  served 
as  the  target  of  operant  training.  Two  spontaneously  active  and  mutually  inhibitory 
neurons  (PGA  and  PGg)  comprised  a  central  pattern  generator  that  drove  the  network 
between  two  ‘behavioral’  or  output  states  (Output  A  or  Output  B).  Each  pattern 
generating  neuron  made  an  excitatory  connection  onto  an  associative  element  (AE).  The 
properties  of  these  associative  elements  were  similar  to  our  model  of  sensory  neuron,  but 
they  were  modified  slightly  in  order  to  reduce  the  transmitter  depletion  that  would 
otherwise  occur  during  the  prolonged  periods  of  activity  during  these  simulations.  These 
associative  elements  were  the  only  elements  in  the  network  that  are  capable  ot  activity- 
dependent  neuromodulation  (i.e.,  associative  enhancement  of  synaptic  strength).  The 
motor  elements  (MNA  and  MNB)  were  driven  by  the  associative  elements.  Activity  in 
the  motor  elements  served  as  the  measure  of  network  behavior,  and  it  fed  back  onto  the 
pattern  generating  elements.  This  excitatory  feedback  from  the  motor  elements 
contributed  to  the  maintenance  of  bursts  of  activity  in  the  pattern  generating  elements. 
The  neural  pathways  for  reinforcement  impinged  on  both  associative  elements.  Positive 
reinforcement,  like  the  US  in  the  simulations  of  classical  conditioning,  facilitated  the 
connection  between  the  associative  and  motor  elements.  This  synaptic  facilitation  was 
enhanced  by  prior  activity  in  the  associative  element  via  the  activity-dependent 
neuromodulation  learning  rule. 

Our  simulations  demonstrated  that  an  associative  element  and  a  learning  rule,  which 
were  originally  derived  in  order  to  simulate  features  of  classical  conditioning,  could  also 
simulate  behavioral  data  on  operant  conditioning.  Specifically,  we  were  able  to 
demonstrate  a  change  in  the  time  spent  performing  a  reinforced  behavior,  and  this 
change  was  dependent  upon  a  contingency  between  the  occurrence  of  that  behavior  and 
the  occurrence  of  reinforcement.  Our  network  also  exhibited  extinction  and  reversal 
learning.  Thus,  the  results  of  these  simulations  illustrate  that,  at  least  in  theory,  the  same 
cellular  mechanisms  that  are  believed  to  underlie  classical  conditioning  in  Aplvsia  could 
also  underlie  operant  conditioning. 

E.  Mathematical  Model  of  a  Bursting  R15  Neuron 

The  R15  cell  in  the  abdominal  ganglion  of  A  ply sia  is  one  of  the  most  widely  studied 
cells  in  the  field  of  electrophysiology.  Many  models  of  this  cell  exist  in  the  literature.  All 
models  attempt  to  mimic  the  endogenous  bursting  behavior  of  R15,  since  it  is  primarily 
this  feature  that  makes  the  cell  interesting.  We  extended  these  earlier  ettorts,  and 
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developed  a  model  of  R15  that:  1)  incorporated  as  much  quantitative  experimental  data 
as  possible  into  the  model  in  a  manner  consistent  with  currently  known  mechanisms;  2) 
provided  a  calcium  ion  balance  within  the  model;  and  3)  simulated  the  electrical  behavior 
of  R15  to  the  fullest  extent  possible.  Our  model  was  a  lumped  parameter  electrical 
equivalent  circuit  (i.e.,  a  Hodgkin-Huxley  type  of  model)  with  twelve  state  variables, 
including  membrane  potential,  internal  calcium  concentration,  calcium  buffer  occupancy 
and  nine  membrane  gating  variables.  On  the  basis  of  available  data,  we  developed  a 
model  that  contains  the  following  components:  inward  currents  (fast  sodium  current, 
/Na ;  background  sodium  current,  /Na  R ;  spiking  calcium  current,  ;  slow  inward 
current,  !S\ )»  outward  currents  (delay  rectifier  current,  /K  ;  calcium-activated  potassium 
current,  5  anomalous  rectifier,  /R  ),  a  non-specific  cation  current  (/NS  ),  and  pump 
and  exchanger  currents  (sodium-calcium  exchanger  current,  /NaCa ;  calcium  pump 
current,  p  ;  sodium-potassium  pump  current,  /NaK ).  The  model  proved  to  be  capable 
of  reproducing  the  bulk  of  the  attributes  of  the  R15  neuron,  including  endogenous 
bursting  modes,  rhythmic  beating  modes,  and  alterations  in  the  waveform  of  the  action 
potential  during  bursting  activity. 

We  also  used  a  model  of  R15  to  study  the  emergence  and  mechanisms  of  chaos  in 
neural  systems.  Chaos  in  neural  systems  can  be  defined  operationally  as  activity  that 
appears  random  but  that  is  generated  by  a  deterministic  system  rather  than  by  additive 
noise.  It  has  been  suggested  that  chaotic  discharge  patterns  may  be  involved  in  such  large 
scale  neuropathological  phenomena  as  seizures,  epilepsy,  tremor  and  convulsions.  In 
addition,  it  has  been  suggested  that  chaos  in  neural  systems  can  have  several  advantages; 
for  example,  biological  chaos  may  help  to  prevent  various  functional  units  in  a  neural 
system  from  becoming  entrained,  or  phase-locked,  into  period  activity.  Our  simulations 
of  the  model  of  R15  exhibited  chaos.  Thus,  chaotic  behavior  can  be  a  results  of  intrinsic 
properties  of  individual  neurons  and  need  not  be  an  emergent  property  of  neural 
assemblies.  Although  the  existence  of  chaotic  behavior  in  R15  has  not  been  examined 
experimentally,  our  modeling  results  predict  that  the  potential  for  such  behavior  exists. 

F.  Identification  of  Neuronal  Elements  of  the  Central  Pattern  Generator  that  Controls 
Feeding  Behavior. 

As  a  first  step  toward  investigating  the  cellular  and  network  properties  that 
contribute  to  operant  conditioning,  we  have  begun  to  identify  elements  of  the  circuit  that 
mediate  feeding  behavior.  Although  much  is  known  about  sensory  neurons,  motor 
neurons  and  modulatory  interneurons  in  the  feeding  motor  system,  up  until  recently,  little 
was  known  about  the  command  neurons  and  pattern  generators  initiating  feeding.  Such 
information  is  critical  for  an  understanding  of  the  neuronal  mechanisms  of  operant 
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conditioning,  since  the  command  and  pattern  generating  neurons  are  possible  loci  for  the 
conditioning. 

Our  approach  was  to  characterize  patterned  neural  activity  in  the  isolated  buccal 
ganglia  and  then  find  cells  initiating  the  patterns.  We  used  identified  cell  B4  and  B5  as 
monitors  of  activity.  We  identified  a  two  neurons,  B31/B32,  whose  activity  seemed  crucial 
to  initiate  rhythmic  firing  in  cells  B4  and  B5.  These  newly  identified  neurons  provide  a 
starting  point  for  investigating  factors  that  initiate  and  control  different  patterns  of  neural 
activity  in  the  buccal  ganglia.  Since  the  buccal  ganglia  are  involved  in  generating  feeding 
behavior,  further  studies  on  the  newly  identified  neurons  may  provide  insights  into  the 
neural  control  of  feeding  behavior,  and  provide  a  neural  substrate  for  studying 
modulation  of  feeding  patterns  by  associative  learning. 
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E.  Books 

Byrne,  J.H.  and  W.O.  Berry  (Eds.).  Neural  Models  of  Plasticity:  Experimental  and  Theoretical 
Approaches,  Orlando,  Academic  Press,  1989. 


V.  Professional  Personnel 

Baxter,  Douglas,  Ph.D. 

Buonomano,  Dean  (Graduate  Student) 

Byrne,  John,  Ph.D. 

Canavier,  Carmen  (Graduate  Student) 

Gingrich,  Kevin,  M.D. 

Raymond,  Jennifer  (Graduate  Student) 

Susswein,  Abraham,  Ph.D. 

VI.  Interactions:  Presentations  to  Professional  Organizations,  Special  Meetings,  and  Invited 

Lectures 

1.  Dr.  Byrne  presented  aspects  of  this  research  at  the  NATO  Advanced  Research  Workshop 
on  Modulation  of  Synaptic  Transmission  and  Plasticity  in  Nervous  Systems,  II  Ciocco,  Italy, 
1987. 

2.  Dr.  Byrne  presented  aspects  of  this  research  at  the  Gif  Lectures  in  Neurobiology  on  the 
Neuronal  Mechanisms  of  Long-Lasting  Changes  in  the  Nervous  System:  Facts  and 
Perspectives,  Gif-sur-Yvette,  France,  1987. 

3.  Dr.  Baxter  presented  the  abstract  "Modulation  of  membrane  currents  and  excitability  by 
serotonin  and  cAMP  in  pleural  sensory  neurons  oiAplysia"  at  the  17th  Annual  Meeting  of 
the  Society  for  Neuroscience,  New  Orleans,  LA,  1987. 

4.  Dr.  Baxter  presented  aspects  of  this  research  at  the  Review  Meeting  for  Air  Force 
Sponsored  Basic  Research,  San  Antonio,  TX,  1987. 

5.  Dr.  Gingrich  presented  the  abstract  "Mathematical  model  of  two  cellular  mechanisms 
contributing  to  dishabituation  and  sensitization  in  Aplysia"  at  the  17th  Annual  Meeting  of 
the  Society  for  Neuroscience,  New  Orleans,  LA,  1987. 
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6.  Dr.  Byrne  presented  aspects  of  this  research  at  the  Bat-Sheva  De  Rothschild  Foundation 
Seminar  on  Neural  Network  Models  and  Their  Relevance  to  Biology ,  Jerusalem,  Israel, 
1988. 

7.  Dr.  Byrne  presented  aspects  of  this  research  at  the  American  Association  for  Artificial 
Intelligence  Symposium  on  Parallel  Models  of  Intelligence :  How  Can  Slow  Components 
Think  so  Fast?,  Stanford,  CA,  1988. 

8.  As  a  speaker  and  conference  co-organizer,  Dr.  Byrne  presented  aspects  of  this  research 
at  the  conference  on  Biotechnology  of  the  Brain:  Fundamental  Discoveries  and  Clinical 
Applications,  Houston,  TX,  1988. 

9.  Dr.  Byrne  presented  aspects  of  this  research  at  the  special  interest  dinner  "Neural 
Networks:  Real  Life  vs  Parallel  Computer"  at  the  18th  Annual  Meeting  of  the  Society  for 
Neuroscience,  Toronto,  Canada,  1988. 

10.  Dr.  Byrne  presented  the  abstract  "Small  networks  of  adaptive  elements  that  reflect  the 
properties  of  neurons  in  Aplysia  exhibit  higher-order  features  of  classical  conditioning"  at 
the  18th  Annual  Meeting  of  the  Society  for  Neuroscience,  Toronto,  Canada,  1988. 

11.  Dr.  Baxter  presented  the  abstract  "Reduction  of  voltage-activated  K+  currents  by 
forskolin  is  not  mediated  by  cAMP  in  pleural  sensory  neurons  of  Aplysia "  at  the  18th 
Annual  Meeting  of  the  Society  for  Neuroscience,  Toronto,  Canada,  1988. 

12.  Dr.  Baxter  presented  the  abstract  "Analysis  and  Simulation  of  Cellular  and  Network 
Properties  Contributing  to  Higher-Order  Features  of  Associative  Learning"  at  the  Cold 
Spring  Harbor  meeting  on  the  Cell  and  Molecular  Neurobiology  of  Aplysia,  1988. 

13.  Dr.  Byrne  presented  aspects  of  this  research  at  the  12th  Harvard  Symposium  on 
Quantitative  Analyses  of  Behavior:  Neural  Network  Models  of  Conditioning  and  Action, 
Cambridge,  MA,  1989. 

14.  Dr.  Byrne  presented  aspects  of  this  research  at  the  Symposium  Medico  Hoechst  23:  The 
Biology  of  Memory,  Munich,  FRG,  1989. 

15.  Dr.  Byrne  presented  aspects  of  this  research  at  the  Gordon  Conference  on  Neuronal 
Plasticity,  Wolfboro,  NH,  1989. 

16.  Dr.  Byrne  presented  aspects  of  this  research  at  the  Second  International  Congress  of 
Neuroethology  Symposium  on  Learning  and  Memory,  Berlin,  FRG,  1989. 
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17.  Mr.  Buonomano  presented  the  abstract  "Simulation  of  Higher-Order  Features  of 
Classical  Conditioning"  at  the  Houston  Bioengineering  Conference,  1989. 

18.  Dr.  Baxter  presented  the  abstract  "Operant  conditioning  can  be  simulated  by  small 
networks  of  neuron-like  adaptive  elements"  at  the  19th  Annual  Meeting  of  the  Society  for 
Neuroscience,  Phoenix,  AZ,  1989. 

19.  Dr.  Byrne  presented  aspects  of  this  research  at  the  Symposium  on  Molluscan 
Neurobiology,  Amsterdam,  Netherlands,  1990. 

20.  Dr.  Byrne  presented  aspects  of  this  research  at  the  Fifth  Annual  Spring  Neuroscience 
Symposium  on  Mechanisms  of  Learning  and  Memory,  Emory  University,  1990. 

21.  As  the  keynote  speaker,  Dr.  Byrne  presented  aspects  of  this  research  at  the  Conference 
on  Activity-Driven  CNS  Changes  in  Learning  and  Development,  State  University  of  New 
York  at  Albany,  1990. 

22.  Dr.  Byrne  presented  aspects  of  this  research  at  the  55th  Symposium  on  Quantitative 
Biology  on  The  Brain,  Cold  Spring  Harbor  Laboratory,  1990. 

23.  As  a  faculty  member,  Dr.  Byrne  presented  aspects  of  this  research  during  the 
Computational  Neuroscience  Course:  Learning  and  Memory,  Cold  Spring  Harbor 
Laboratory,  1990. 

24.  Dr.  Byrne  presented  aspects  of  this  research  at  the  Cold  Spring  Harbor  meeting  on  the 
Cell  and  Molecular  Neurobiology  ofAplysia,  1990. 

25.  Dr.  Baxter  presented  the  abstract  "Mathematical  modeling  of  the  serotonergic 
modulation  of  electrophysiological  properties  of  sensory  neurons  in  Aplysia"  at  the  Cold 
Spring  Harbor  meeting  on  the  Cell  and  Molecular  Neurobiology  ofAplysia,  1990. 

26.  Dr.  Baxter  presented  the  abstract  "Mathematical  modeling  of  the  serotonergic 
modulation  of  electrophysiological  properties  of  sensory  neurons  in  Aplysia"  at  the  20th 
Annual  Meeting  of  the  Society  for  Neuroscience,  St.  Louis,  MI,  1990. 

27.  Ms.  Raymond  presented  the  abstract  "Activity-dependent  neuromodulation  can  support 
operant  conditioning  in  a  small  oscillatory  network",  at  the  Cold  Spring  Harbor  meeting 
on  the  Cell  and  Molecular  Neurobiology  ofAplysia,  1990. 
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VII.  New  Discoveries  and  Specific  Applications 

A  number  of  scientific  discoveries  resulted  from  the  research  sponsored  by  the  AFOSR 
Grant  87-0274.  First,  we  found  that  in  tail  sensory  neurons  serotonin  modulates  not  only  /j^s 
but  that  it  also  modulates  I^y.  Since  the  serotonergic  modulation  of  these  sensory  neurons  is 
believed  to  contribute  to  simple  forms  of  learning  in  Aplysia,  this  discovery  provided  new 
insight  into  the  cellular  mechanism  of  learning.  Second,  we  identified  neurons  that  are 
elements  of  the  central  pattern  generator  that  controls  feeding  behavior  in  Aplysia.  Feeding 
behavior  in  Aplysia  has  been  shown  to  be  modified  by  operant  conditioning  like  training,  and 
before  a  cellular  analysis  of  the  cellular  mechanisms  of  operant  conditioning  can  begin  it 
necessary  to  identify  the  neurons  underlying  the  behavior.  Thus  the  identification  of  the 
pattern  generating  neurons  is  an  important  first  step  in  the  investigation  of  the  cellular 
mechanisms  of  operant  conditioning.  In  addition,  our  simulations  have  led  to  the  development 
of  biologically  derived  computational  algorithms  that  simulate  many  features  of  nonassociative 
and  associative  learning.  Although  it  is  too  early  in  the  research  to  comment  on  specific 
applications  of  this  research,  these  results  are  relevant  to  aspects  of  artificial  intelligence  and 
provide  a  starting  point  for  developing  machines  having  some  of  the  information  processing 
capabilities  of  the  nervous  system. 

No  inventions  were  made. 
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